Diagnostic Performance of Xpert MTB/RIF Ultra Compared with Predecessor Test, Xpert MTB/RIF, in a Low TB Incidence Setting: a Retrospective Service Evaluation

ABSTRACT The aim of this study was to evaluate the performance of Xpert MTB/RIF Ultra (Ultra) compared with its predecessor, Xpert MTB/RIF (Xpert), in the diagnosis of tuberculosis (TB) in a low TB incidence country. Retrospective analysis was performed on 689 clinical samples received between 2015 and 2018, on which Xpert was performed, and on 715 samples, received between 2018 and 2020, on which Ultra was performed. Samples were pulmonary (n = 830) and extrapulmonary (n = 574) in nature, and a total of 264 were culture positive for Mycobacterium tuberculosis complex (MTBC). The diagnostic performance of both assays was analyzed using culture as the reference standard. The sensitivity of Ultra for culture positive (smear positive and smear negative) MTBC samples, was 93.2% (110/118) compared with 82.2% (120/146) for Xpert (P = 0.0078). In smear negative-culture positive samples, Ultra had a sensitivity of 74.2% (23/31) versus 36.11% (13/36) for Xpert (P = 0.0018). Specificity of both assays was comparable at 94.8% (566/597) for Ultra and 95.8% (520/543) for Xpert (P = 0.4475). The sensitivity of Ultra and Xpert assays among exclusively pulmonary samples was 95.3% (82/86) and 90.3% (84/93), respectively (P = 0.1955), and 87.5% (28/32) and 67.9% (36/53), respectively, among extrapulmonary samples (P = 0.0426). Ultra showed improved performance compared with Xpert in a low TB incidence setting, particularly in smear negative and extrapulmonary MTBC disease. The specificity of Ultra was lower than Xpert, however, this was not statistically significant. IMPORTANCE The study demonstrates the improved sensitivity of the Ultra compared with the Xpert, particularly in smear negative TB disease, for both pulmonary and extrapulmonary samples in a low TB incidence setting. Cycle threshold (Ct) value for both assays was found to positively correlate with time to TB culture positivity, suggesting that Ct and semiquantitative results could be used as indicators of sample MTBC bacillary burden, and thus, perhaps, of transmission potential. This may have implications for the designation of patient isolation precautions.

1. Findings from this study are comparative with previously published papers demonstrating the increased sensitivity for Ultra, with a slightly lower specificity. Some of these publications (with a 2019 publication by Pocognoli et al as example) have also shown these results in a low TB incidence setting. Given this overlap, it is suggested to the authors that additional details be added regarding potential impact of Ultra implementation in their region and the meaning and impact of "Trace" calls (which needs to be further expounded upon) to further the impact of test data amidst previous publications. 4. Note that in some countries (South Africa, for example), Xpert Ultra has replaced smear microscopy as an initial test. This should be added to the discussion. 5. Line 148: Change to Cape "Town" 6. Line 227: Change "Fitzgibbon" Reviewer #2 (Comments for the Author): In this manuscript, "Diagnostic performance of Xpert MTB/RIF Ultra® compared with predecessor test, Xpert MTB/RIF®, in a low TB incidence setting: a retrospective service evaluation", the authors, Mary Mansfield et al., show that Xpert MTB/RIF Ultra has a higher sensitivity but lower specificity than the previous method Xpert MTB/RIF in detecting M. tuberculosis in pulmonary and extrapulmonary samples. The phenotypic DST did not detect resistance potentially caused by some rpoB mutations; this should have been discussed. The manuscript was rather well structured but would have gained on being shorter. #1 Line 1 (Abstract) and Line 49 (Introduction): The words "MTBC infection" is typically used for latent TB, not for active tuberculosis, which is diagnosed with the methods described in the manuscript.
#2 Methods Lines 51 -56: I miss a brief description of the methods used, both for Xpert and Ultra and for culture. How were sample types not mentioned in the Xpert/Ultra Cepheid's Instructions for Use treated? How were samples with small specimen volumes treated? As far as I can understand, the study was done on all routine samples 2015 --2020 positive or negative for MTBC in culture and had an Xpert or Ultra result; however, this is not stated. Which samples types were/were not analyzed routinely?
#3 Methods Lines 60 -64: How many samples were "invalid"? Were any other samples excluded, except those mentioned in the text? How were microscopy-positive samples with the growth of non-tuberculous mycobacteria handled in the study? #10 Discussion, Lines 145 -150: Please speculate what you think the causes for the differences in specificities between Xpert and Ultra might be. #11 Discussion, Lines 151 -159: Please discuss why the differences in specificities between Xpert and Ultra are less in your low incidence setting than in a high incidence setting.

Preparing Revision Guidelines
To submit your modified manuscript, log onto the eJP submission site at https://spectrum.msubmit.net/cgi-bin/main.plex. Go to Author Tasks and click the appropriate manuscript title to begin the revision process. The information that you entered when you first submitted the paper will be displayed. Please update the information as necessary. Here are a few examples of required updates that authors must address: • Point-by-point responses to the issues raised by the reviewers in a file named "Response to Reviewers," NOT IN YOUR COVER LETTER. • Upload a compare copy of the manuscript (without figures) as a "Marked-Up Manuscript" file. • Each figure must be uploaded as a separate file, and any multipanel figures must be assembled into one file. For complete guidelines on revision requirements, please see the journal Submission and Review Process requirements at https://journals.asm.org/journal/Spectrum/submission-review-process. Submissions of a paper that does not conform to Microbiology Spectrum guidelines will delay acceptance of your manuscript. " Please return the manuscript within 60 days; if you cannot complete the modification within this time period, please contact me. If you do not wish to modify the manuscript and prefer to submit it to another journal, please notify me of your decision immediately so that the manuscript may be formally withdrawn from consideration by Microbiology Spectrum.
If your manuscript is accepted for publication, you will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
Corresponding authors may join or renew ASM membership to obtain discounts on publication fees. Need to upgrade your membership level? Please contact Customer Service at Service@asmusa.org.
Thank you for submitting your paper to Microbiology Spectrum.

Reviewer #1 (Comments for the Author):
The authors provide an evaluative comparison study of the two versions of GeneXpert (MTB/RIF and Ultra) to better understand diagnostic performance in a low TB incidence setting.
1. Findings from this study are comparative with previously published papers demonstrating the increased sensitivity for Ultra, with a slightly lower specificity. Some of these publications (with a 2019 publication by Pocognoli et al as example) have also shown these results in a low TB incidence setting. Given this overlap, it is suggested to the authors that additional details be added regarding potential impact of Ultra implementation in their region and the meaning and impact of "Trace" calls (which needs to be further expounded upon) to further the impact of test data amidst previous publications Please see lines 176-182.
2. The difference in Xpert negative vs Ultra negative samples is striking (11.5% vs 0%), and should be discussed.
Wording clarified line 119-123. This reflects improved sensitivity of Ultra and its ability to detect MTBC even in smear negative samples.
3. Time to positivity (TTP) can differ between liquid and solid media, this should be expounded upon.
TTP was determined from growth on liquid media. Line 68-70.
4. Note that in some countries (South Africa, for example), Xpert Ultra has replaced smear microscopy as an initial test. This should be added to the discussion. Please see line 188. 5. Line 148: Change to Cape "Town" Amended. 6. Line 227: Change "Fitzgibbon" Amended.

Reviewer #2 (Comments for the Author):
In this manuscript, "Diagnostic performance of Xpert MTB/RIF Ultra® compared with predecessor test, Xpert MTB/RIF®, in a low TB incidence setting: a retrospective service evaluation", the authors, Mary Mansfield et al., show that Xpert MTB/RIF Ultra has a higher sensitivity but lower specificity than the previous method Xpert MTB/RIF in detecting M. tuberculosis in pulmonary and extrapulmonary samples. The phenotypic DST did not detect resistance potentially caused by some rpoB mutations; this should have been discussed. The manuscript was rather well structured but would have gained on being shorter. Amended.
#1 Line 1 (Abstract) and Line 49 (Introduction): The words "MTBC infection" is typically used for latent TB, not for active tuberculosis, which is diagnosed with the methods described in the manuscript. Amended.
#2 Methods Lines 51 -56: I miss a brief description of the methods used, both for Xpert and Ultra and for culture. How were sample types not mentioned in the Xpert/Ultra Cepheid's Instructions for Use treated?
Details of methods added. All samples (both pulmonary and extra-pulmonary) were processed the same way for Xpert and Ultra tests.
How were samples with small specimen volumes treated? All sample types were decontaminated prior to processing on the Xpert assays. 0.5 ml of the decontaminated sample was added to 1.5 ml of Sample Reagent and added to the cartridge.
As far as I can understand, the study was done on all routine samples 2015 --2020 positive or negative for MTBC in culture and had an Xpert or Ultra result; however, this is not stated. Which samples types were/were not analyzed routinely?
Please see lines 57-79.  Table 1. I have moved this to the supplementary section. The figure represents a graphical representation using ROC curves of the PPV/NPV data which the journal's readers may find helpful when read in conjunction with the data shown in Table 1. #5 Results Lines 93 -99: I find it challenging to understand by the text which semiquantitative values are significantly different from TTP. Maybe this could be illustrated in Figure 2?
Adding extra notations to the graph made it difficult to read.
#6 Figure 2 would, in my view, look better if both y-axes had the same scale and each category on both x-axes were equally broad. Indicate, if possible, the significant differences.
Agreed, but when inputting the data into GraphPad Prism, due to the significant difference in ranges between the 2 data sets, the graphs were difficult to compare to each other.  Your manuscript has been accepted, and I am forwarding it to the ASM Journals Department for publication. You will be notified when your proofs are ready to be viewed.
The ASM Journals program strives for constant improvement in our submission and publication process. Please tell us how we can improve your experience by taking this quick Author Survey.
As an open-access publication, Spectrum receives no financial support from paid subscriptions and depends on authors' prompt payment of publication fees as soon as their articles are accepted. You will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data. If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record. If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication of your article may be delayed; please contact the ASM production staff immediately with the expected release date.